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ABSTRACT 



This paper discusses the lessons learned from the use of 
early state-of-the-art Computer-Based Training (CBT) applications and hopes 
for the future. The paper is based on experiences with large introductory 
personal computer applications classes with over 700 students that used CBT 
such as Virtual Tutor and Pinpoint . Findings are presented related to student 
evaluation of the CBT system, including student opinions of Pinpoint, the 
class World Wide Web page, the textbook, and how much was accomplished in the 
course, as well as whether more assistance from the instructors was required 
and whether they would like to learn more about programming and/or using 
software applications. The following conclusions reported by instructors are 
also presented: (1) the interaction with Pinpoint proved to contribute to 

successful learning; (2) CBT can be used effectively in small or large 
classes if it is customizable; (3) CBT eases students' inhibitions about 
using and learning technology; and (4) CBT can be an efficient learning tool 
in a classroom without taking away the vital importance of the instructor. 
(Contains 10 references.) (MES) 
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Abstract: Compirters are established today as a part of our day to day life. They are no 
longer just a professional tool but rather a commodity that helps people to work, 
communicate, shop or trade on line and at last, but not least, to improve the educational 
p^s. In this paper, we wiU discuss the lessons learned fiom our use of early state of the 
ait Computer-Base Training (CBT) applications and what we hope to see in the future Our 
^sen^on IS on the experience of the first author with large classes with over 700 
studmts that UMd CBT like Virtual Tutor and PinPoint Although our experiences are tied to 
specific ^Is, tte lessMis learned can be applied to a wide variety of similar tools Actuanv 
^ tools tend to be more similar than not, which is rather disj^ipointing considering there 
IS not one approved way of “doing it right” ^ 



Introduction 

tool b.H as a part of our day to day life. They are no longer just a professional 

to^oT T?® fT computers suggested the obvious educational application: automated training and 

computer supported training resulted in some early products that 
were not based on i«dagopcal theories (Docent, 1998), (VirtualTutor. 1999). The results w^^t very 

S"v^t^’ga ^ ^ improving since then, yet there is stiul 

of continuos increase in demand and the high level of expectations in the educational sector 

® ^ CBT applications, more and more companies are 
ZSLn of fT® *ese type of applications. Actually, these tools tend to be more 

^ar thm not, which is rathCT disappomtmg considering there is not one approved way of “doing it right.” 
Howwer tos large selection of CBT applications can rather confuse than help educators in choosing the “right 
theoi^cal and jnactical experience regarding CBT applications we address some of the 

CT?a^li^om developments of 
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The Past 



In the early years of CBT, the approaches were rather technology-driven using simple ^hnology md 
w=,Toto pteMd or by with mst™tiom™. 11.= CBT t^pbc^ons « no. pjo^de 

S i^on and tedbnok was laoking n> help snnfents judge die effecweness of to lea^. Oto 
these systems were simply psesenting a compnterized version of a textbook, which often is mnch 1^ meM 

to tSL hook. A^y, dns appreaoh is sfill mkeo toy by some publishers bo. oan result m d«ashons 

™The early systems were also lacking tools for managing the classroom. They did not incto the shtots’ 
reconis of gt^ and actiyities. They weren’t portable but tied to a speoiiio platfonn or networked. Smgle user 
systems available on floppy disks and later on CD-ROMs was the state of the art. 



The Present 

Many tools have been improved pedagogically (CBT Systems, 1999). They are totive and help &e 
stud^tefuS their own progress and^ effective their learning is. However, smce web based appro^hes 
itonable now, we need to stress that these systems stiU tend to be inappropnate to support the 

leanmgF^s. systems classroom in various ways. Mmy of die ^ 

networked and Ime many integrated functions. The systems record the dates when the J^on^ 
and also automatically grade multiple-choice tests and record the results. Thus, they help the instructore “^k 
over the students’ shoulder” to see if they are doing the work and if the students are on track, e.g., by checkmg 
how much time the students spend on taking a tutorial and how good their test results are. . .^ 

Unfortunately, these systems are only available for introductory courses that do not 
flexibility. Also, Aey are often not comprehensive and stable enough to run easily 
Furthermore, the procedures for setup and the maintenance are quite complica^ So it mght c 

find the faculty and staff to run such a system in a non-technical department. There are also no g(^ 
creating educationally effective web sites. The ^sterns 
on some pretest and they don’t take much advantage of the possibihties of hy^edia. They ^on fw 
much exp^tion, i.e., they are quite linear forcing the students to go along with one of the few predetermmed 

^^'^More and more systems are working in a simulated environment (in part to elimimte ^y 
incompatibilities between the lab networks and/or computers configurations and the * 

svsteiS Another reason for a strong trend in developing these type of systems is the 
sS2 to use it on their own personal computers without having to purchase the required software 

applirafaoas ^ movement toward developing integrated learning environment including training 

evaluating, assessment and testing computer base applicatioa These integrated le^g 
Server-based or Web-based and tend to be pedagogically inadequate. Where^Ae technology migh 
moved one step forward, the pedagogy of went two steps backwards (Mioduser, 1999). 



The Future 

So what would we like to see in the future based on our experience with Aese 
based on studying some related issues from a more research point of view (Hmelo, 1996, Hubscher, 1997, 

We would like to see more adaptive tutorials based on preassessment and other f^^k tom the ^ent 
with the goal of keeping the student in the zone of proximal development (Vygotsky, 1962). This would re^ 
in more ^individualized courses, more effective learning, and allow more self-directed path through the 
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toSStos '* development, it ndJI also remit In a ntote 

imeresting and less frustrating learning expenence for the student. 

editoreSy^do^t educationally effective presentations is desired. HTML 



Findings 

n «T application (Kelly Services Co, 1 999) before usine it for 

qu^er. At the end of fte quarter we used on line class evaluation involving students of frie pilot test section 
^ a te^ar ^on of tie oonrse as conttol gtnttp. lleso aro the results of L evaluati™, ' 

Student Previous Computer Experience uauuu. 

aavtai *! "“-“Steering students with relatively little computer experience The self- 

n of students’ previous computer experience was lower for the for the regular section 

1.60) ti^for the Pinpoint s^on (2.05). figure 1 shows the results of the students' self-^vXSrT^ 
scale of 1 two five, for the level of previous computer experience (in percentage). 




^^ce for the level of previous computer 



Student Evaluation 



had used the CBT system, we asked for their feedback. The questions we asked were- 

• How would you rate your opinion of Pinpoint? 

• How would you rate your opinion of the COMPOlOO WEB PAGE? 

• How would you rate your opinion of COMPOl 00 TEXTBOOK? 

• How would you rate your opinion of how much you have accomplished in COMPOlOO? 

signiSt. ^ ^ CBT, it is statistically not 
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Figure 2: Chart rqjresenting the results of the Student Evaluation 

We asked two more questions to find out how valuable they thought the PinPoint tool was with the 
following two questions. The results can be seen in Figure 3. 

• Do you think you need more assistance from your instructor is required? 

• Would you like to learn, in the near future, more about programming and/or using software 




i Instructor Programming Using j 

! _ Software 



Figure 3 : Chart rq)resenting the results of the Student Evaluation 



Instructor’s evaluation 

After using PinPoint in one section (out of a total of eighteen sections) we have ^ed ming 
in aU eighteen sectfons. Below are the conclusions reported by approximately to mstructois 
We are plig to collect more data to find out whether the instructors' subjec^e reports match the ^ 
^rformance of die students. The data reported is statistically not significant. The instructors are gradua 
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students of the Computer Science and Software Engineering department. 

1 . The interaction with PinPoint proved to contribute to successful learning (reading, typing, using the 
mou^, mswemg questions, and thinking). However, the use of too much video in a CBT pr^uct 
caused the students’ bram to go into passive mode, thus reducing the effectiveness of the CBT : 

fte average grade of the students using a more dynamic & interactive upgraded CBT was 
higher 

students using the upgraded CBT were more independent in doing the hands on projects so 
instructors could spend more time on pedagogical issues 

effectively in smaU or large classes if they are customizable (or even minimally 

^ptive). ^ are good m creating richer learning environments and allowing students to learn at 
tneir OMim speed r 

a customable CBT helps instructors to create training modules that better match the 
stunts b^kground and goals, the textbooks (or concepts book) used, and the training 
modules and hands on projects selected 

• based on the results of a pre-test, a customizable CBT can help instructors create different 
leammg paths for students helpmg students to be more effective. This includes 



2 . 



a self-paced multimedia approach for the students with an “above average” 
computer backgroimd 

or a “CBT instructor-led classroom” approach for the students with a “below 



3. CBT eases students’ inhibitions about using and learning technology : 

• by helpmg ^ents to become more independent in lab and also increasing their self-esteem 

a g^ CBT can also reduce students’ inhibitions about using computers and their software 
applications 

4. CBT can be very efficient learning tools in a classroom without taking away the vital importance of 

the mstructor : ■' *' 



tile results of the survey show clearly that students felt strongly about the necessity of 
mstructor assistance 

based the detailed training reports provided by the management component of a 
competitive CBT, (see Figure 4) instructors can realize which subjects represent a real 
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Figure 4 : PinPoint detailed view of a t raining report 



Conclusions 



We have been somewhat critical about the past and present CBT systems. However, we feel that quite a 
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few systems are on the right track but are still lacking many elements directly rela^ to pedagogy. For 
somewhat understandable reasons, these parts that are implemented are not difficult to do m prmciple, 

f»cr»priallv helDine managing of classes and now also networking. 

NevCTtheless, an experiment in our 700-plus student classes to figure out the effectiveness of a CBT 
system (PinPoint) showed quite favorable results and we will keep continumg usmg CBT systems for our large 

mtro^to^t“of the experiment suggest the following. Interaction with the system (readmg, pTmg. 

the mouse and answering questions) contributed to successful leammg. The use of too mv^h video “ » CBT 

product causes students to go into a passive hands-off mode reducing its effectiveness. CBT 

?ear of using and learning with computer technology. This is apparently still a problem for many students- 

not all ofthem grow up with a computer under the pillow. 

CBT can iJa very efficient learning tool in classroom without takmg away the vital mportance of ffie 
instructor. We beUeve that CBT can be used even more effectively m small and large clas^^ if they 
more adaptive to the individual student and provide a richer leammg envuonment allowmg the 
students to leam in a more individualized fashion supporting the students needs. 
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